Remove SQLite and deploy Postgres by default#1503
Merged
Conversation
Signed-off-by: Jeremy Alvis <jeremy.alvis@solo.io>
Signed-off-by: Jeremy Alvis <jeremy.alvis@solo.io>
Signed-off-by: Jeremy Alvis <jeremy.alvis@solo.io>
Contributor
|
Please consider support https://github.com/cloudnative-pg/cloudnative-pg |
Signed-off-by: Jeremy Alvis <jeremy.alvis@solo.io>
EItanya
approved these changes
Mar 16, 2026
Contributor
Author
|
Follow-up to this in: #1527 |
EItanya
pushed a commit
that referenced
this pull request
Mar 23, 2026
# Description Follow-up to: #1503 > [!WARNING] > If you are using a vector enabled DB for your external database, set `database.postgres.vectorEnabled` to `true` as the default value has changed to `false` in order to use the official `Postgres` image instead of the `pgvector` image for the bundled database. > > If you were using the bundled PostgreSQL and want to keep the data, read the mitigation section below. A direct upgrade will initialize a fresh database. This PR reworks how kagent's bundled PostgreSQL is configured and deployed. The main goals are: - **Security:** credentials are now stored in a Kubernetes Secret instead of a ConfigMap - **Reliability:** the bundled pod now has liveness/readiness probes, a `Recreate` strategy, non-root security context, and a correctly configured `PGDATA` path that survives restarts - **Flexibility:** `bundled.enabled` and `url`/`urlFile` are now independent — you can keep the bundled pod running while pointing the controller at an external database, which makes migration easier - **Clarity:** the bundled instance is more explicitly scoped to dev/eval use; the image is switched to standard `postgres:18` in the helm chart and the toggle is an explicit flag rather than an implicit side effect of leaving `url` empty. The local make target deploys `pgvector:pg18-trixie` for developing against a vector enabled database. Note that this is a breaking change from the bundled Postgres added earlier this week. ### What changed in the helm chart for the bundled image: | | Before | After | |---|---|---| | Image | `pgvector/pgvector:pg18-trixie` | `docker.io/library/postgres:18` | | Username | `postgres` | `kagent` (hardcoded) | | Database | `postgres` | `kagent` (hardcoded) | | Mount path | `/var/lib/postgresql` | `/var/lib/postgresql/data` | | PGDATA | `/var/lib/postgresql/data` (default) | `/var/lib/postgresql/data/pgdata` (explicit) | | Password storage | ConfigMap (plaintext) | Secret (base64) | | Toggle | implicit (empty `url`/`urlFile`) | `database.postgres.bundled.enabled` (default: `true`) | Restarts and helm upgrades preserve data in the bundled Postgres instance correctly once on the new chart. ### Mitigation The bundled PostgreSQL is intended for local development. If you don't need to keep existing data, just upgrade — the new chart initializes a fresh database at a different path on the existing PVC and will not touch the old data directory. If you want to keep existing local data, you'll need to back up first and restore after upgrading. Backup (run before upgrading): ```bash kubectl exec -n kagent deployment/kagent-postgresql -- \ pg_dump -U postgres postgres > kagent-backup.sql ``` Restore (run after upgrading): **Note** The helm chart by default uses the `Postgres` image without vector support whereas the `helm install` make target overwrites the image to use the `pgvector` image. Restoring when overwriting the bundled image to use the `pgvector` image: ```bash PGPOD=$(kubectl get pod -n kagent -l app.kubernetes.io/component=database -o name | head -1) kubectl exec -i -n kagent $PGPOD -- psql -U kagent -d kagent < kagent-backup.sql ``` ### Database configuration reference `bundled.enabled` and `url`/`urlFile` are independent controls: - **`database.postgres.bundled.enabled`** controls whether the bundled PostgreSQL pod and its PVC are deployed. It has no effect on which database the controller connects to. - **`database.postgres.url` / `database.postgres.urlFile`** control what the controller connects to. When either is set, the controller uses it. When both are empty, the controller connects to the bundled instance. This means you can have the bundled pod running while the controller points at an external database — useful for migrating data from the bundled Postgres to an external Postgres. **Connection precedence (controller only):** `urlFile` > `url` > bundled connection string. | Scenario | `bundled.enabled` | `url` / `urlFile` | Bundled pod deployed? | Controller connects to | |---|---|---|---|---| | Default (dev/eval) | `true` | unset | yes | bundled | | External DB, no bundled pod | `false` | set | no | external | | External DB, bundled pod kept running | `true` | set | yes | external | | Bundled disabled, no external set | `false` | unset | no | nothing (misconfigured) | **urlFile** is recommended when your connection string contains credentials — it keeps secrets out of Helm values and the Kubernetes Deployment spec: ```yaml database: postgres: urlFile: /var/secrets/db-url # path inside the controller container ``` Mount the secret yourself via `controller.volumes` / `controller.volumeMounts`: ```yaml controller: volumes: - name: db-secret secret: secretName: my-postgres-url-secret volumeMounts: - name: db-secret mountPath: /var/secrets readOnly: true ``` **url** is suitable when credentials are already managed externally (e.g. injected by a secrets manager at deploy time): ```yaml database: postgres: url: "postgres://user:pass@db.example.com:5432/kagent?sslmode=require" ``` **bundled** deploys a single-replica PostgreSQL pod with a PersistentVolumeClaim. Not suitable for production — no replication, no backups, data is lost if the PVC is deleted: ```yaml database: postgres: bundled: enabled: true # default storage: 500Mi image: registry: docker.io repository: library name: postgres tag: "18" ``` --------- Signed-off-by: Jeremy Alvis <jeremy.alvis@solo.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Removes SQLite as a supported database backend and makes PostgreSQL the only supported option. Bundles a PostgreSQL instance (with pgvector) in the Helm chart so that
helm installworks out of the box with no external prerequisites.Closes: #1502
Changes
Helm
helm/kagent/templates/postgresql.yaml— deployed whendatabase.postgres.urlanddatabase.postgres.urlFileare both empty (the default)database.postgres.urlto skip the bundled instance and use an external PostgreSQLdatabase.type,database.sqlite, SQLite emptyDir volume, andXDG_CACHE_HOMEfrom the controller deploymentCI
strategy.matrix: database: [sqlite, postgres]fromtest-e2e— single database, no matrix neededmake helm-installHelm values
Test plan
go test -race -skip 'TestE2E.*' ./...)helm unittest helm/kagent)TestE2EMemoryWithAgentpasses — confirms pgvector and memory table work end-to-endmake helm-install push-test-agent push-test-skillfollowed by e2e tests viago test -v github.com/kagent-dev/kagent/go/core/test/e2e -failfast -shuffle=on)